1. Field of the Invention
The present invention relates generally to an improved data processing system and method. More specifically, the present invention is directed to mechanisms for management of an input/output virtualization (IOV) adapter, such as a Peripheral Component Interconnect (PCI) IOV adapter, through a virtual intermediary in a hypervisor and through a management program in an IOV management partition. More specifically, the present invention relates to the sharing of an IOV adapter across multiple systems with a multi-root PCI manager (MR-PCIM) being implemented in a logical partition (LPAR) on at least one of the systems that is potentially sharing the IOV adapter.
2. Background of the Invention
Most modern computing devices make use of input/output (I/O) adapters and buses that utilize some version or implementation of the Peripheral Component Interconnect standard, which was originally created by Intel in the 1990s. The Peripheral Component Interconnect (PCI) standard specifies a computer bus for attaching peripheral devices to a computer motherboard. PCI Express, or PCIe, is an implementation of the PCI computer bus that uses existing PCI programming concepts, but bases the computer bus on a completely different and much faster serial physical-layer communications protocol. The physical layer consists, not of a bi-directional bus which can be shared among a plurality of devices, but of single uni-directional links, which are connected to exactly two devices.
FIG. 1 is an exemplary diagram illustrating a system that incorporates a PCI Express (PCIe) bus in accordance with the PCIe specification. The particular system shown in FIG. 1 is a blade enclosure in which a plurality of server blades 101-104 are provided. A server blade is a self-contained computer server designed to for high density systems. Server blades have many components removed for space, power and other considerations while still having all the functionality components to be considered a computer. The blade enclosure 100 provides services, such as power, cooling, networking, various interconnects, and management of the various blades 101-104 in the blade enclosure 100. The blades 101-104 and the blade enclosure 100 together form a blade system.
As shown in FIG. 1, PCIe is implemented on each of the server blades 101-104 and is used to connect to one of the PCIe devices 105-112. Each of these server blades 101-104 is then plugged into a slot in the blade enclosure 100 which then connects the outputs of the PCIe Ethernet devices 105, 107, 109, and 111 to an Ethernet switch 113, via a backplane in the blade enclosure 100, which then generates Ethernet connections 115 for external connectivity, i.e. communication connections to devices outside the blade enclosure 100. Similarly, each of the PCIe storage devices 106, 108, 110, and 112 are connected via the backplane in the blade enclosure 100 to a storage area network switch 114 which then generates storage area network connections 116 for external connectivity.
Thus, the system shown in FIG. 1 is exemplary of one type of data processing system in which the PCI and/or PCIe specifications are implemented. Other configurations of data processing systems are known that use the PCI and/or PCIe specifications. These systems are varied in architecture and thus, a detailed treatment of each cannot be made herein. For more information regarding PCI and PCIe, reference is made to the PCI and PCIe specifications available from the peripheral component interconnect special interest group (PCI-SIG) website at www.pcisig.com.
In addition to the PCI and PCIe specifications, the PCI-SIG has also defined input/output virtualization (IOV) standards for defining how to design an I/O adapter (IOA) which can be shared by several logical partitions (LPARs). A LPAR is a division of a computer's processors, memory, and storage into multiple sets of resources so that each set of resources can be operated independently with its own operating system instance and applications. The number of logical partitions that can be created depends on the system's processor model and resources available. Typically, partitions are used for different purposes such as database operation, client/server operation, to separate test and production environments, or the like. Each partition can communicate with the other partitions as if the other partition is in a separate machine. In modern systems that support LPARs, some resources may be shared amongst the LPARs. As mentioned above, in the PCI and PCIe specification, one such resource that may be shared is the I/O adapter using I/O virtualization mechanisms.
Further, the PCI-SIG has also defined IOV standards for sharing IOAs between multiple systems. This capability is referred to as multi-root (MR) IOV. FIG. 2 is an exemplary diagram illustrating a system incorporating a PCI Express (PCIe) MR IOV. In particular, FIG. 2 illustrates how the architecture shown in FIG. 1 can be modified in order to share the PCIe devices across multiple systems. The server blades 201-204 now generate PCIe root ports 205-212 and drive PCIe connections across the blade enclosure 200 backplane, instead of incorporating the PCIe devices themselves on the sever blades 201-204 as was done with the server blades 101-104 in FIG. 1. The PCIe links from each server blade 201-204 are then connected to one of the multi-root (MR) PCIe switches 213-214 which are in turn connected to the PCIe devices 217-220. The PCIe devices 217-220 connect to the external Ethernet and storage devices through the external connectivity 215 and 216. Thus, PCIe devices can be used within the blade enclosure 200. This reduces overall costs in that the number of PCIe devices 217-220 may be minimized since they are shared across server blades 201-204. Moreover, this may reduce the complexity and cost of the server blades 201-204 themselves by not requiring integration of the PCIe devices 217-220. While the PCI-SIG provides a standard for defining how to design an IOA which can be shared by several LPARs, this specification does not define how to connect the IOA into a host system. Moreover, the standard does not specify how to manage the shared functionality of an IOA utilizing I/O virtualization. The standard also does not specify how to manage the MR I/O fabric, for example MR fabric 221 in FIG. 2, when MR IOV is implemented. This is because the PCI-SIG specification is concerned with setting standards for the operation of the PCIe fabric below the root complex. In other words, the PCI-SIG does not provide any definition of standards for the root complex and above because that is considered the domain of system houses. That is, each of an Intel platform, an IBM Power® platform, and a Sparc platform, for example, may have different system implementation requirements that are not set forth in the PCI-SIG standards.